An Expert System for Recognition of Facial Actions and their Intensity

The Facial Action Coding System (FACS) is an objective method for quantifying facial movement in terms of 44 component actions, i.e. Action Units (AUs). This system is widely used in behavioral investigations of emotion, cognitive process and social interaction. Highly trained human experts (FACS coders) presently perform the coding. This paper presents a system that can automatically recognize 30 AUs, their combinations and their intensity. The system employs a framework for hybrid facial feature detection and an expert system for facial action coding in static dual-view facial images. Per facial feature, multiple feature detection techniques are applied and the resulting redundant data is reduced so that an unequivocal facial expression geometry ensues. Reasoning with uncertainty is used to encode and quantify the encountered facial actions based on the determined expression geometry and the certainty of that data. Eight certified FACS coders tested the system. The recognition results demonstrated rather high concurrent validity with human coding.